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PMO  Role  in  Systems  Engineering  i 


CMMI 


Inherent  PMO  Responsibility: 

•  Ensure  technology  readiness  level  is  appropriate  for  program 
phase 

•  Develop  initial  system  requirements  in  conjunction  with 
stakeholders  and  ensure  continued  involvement 

•  Develop  technical  evaluation  criteria  and  evaluate  proposals 
during  source  selection 

•  Develop  independent  cost  and  schedule  estimates  for  the 
technical  effort 

•  Ensure  external  interfaces  are  properly  identified  and 
monitored 

•  Ensure  PMO  has  adequate  systems  engineering  staff 
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PMO  responsibility  in  conjunction  with  your  contractor: 

•  Ensure  contractor  development  method  is  appropriate 

•  Ensure  contractor’s  systems  engineering  processes  are 
acceptable  and  being  followed 

•  Ensure  compatible  processes  between  prime  and  sub 
contractors  and  between  the  contractor  team  and  the  PMO 

•  Review  and  approve  systems  engineering  documentation 

•  Ensure  systems  engineering  function  is  adequately  integrated 
with  other  areas  such  as  logistics  and  test 

•  Manage  the  top-level  change  control  process 

•  Perform  technical  evaluations 

•  Systems  Integration  (if  applicable) 

•  Ensure  end  system  meets  requirements 
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Understanding  Engineering  PAs 

The  Engineering  process  areas  establish  a  consistent  set  of 
requirements  that  are  derived  from  stakeholder  needs  and 
operational  capability  statements  so  that  work  products 
developed  internally  by  the  acquirer  and  work  products  and 
delivered  systems  from  the  suppliers  are  proven  to 
successfully  satisfy  end  user  needs  and  provide  operational 
capabilities. 


*  Requirements  Management  REQM 

*  Requirements  Development  RD 

•  Verification  VER 

•  Validation  VAL 
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The  purpose  of  requirements  development  is  to  produce 
and  analyze  customer,  product,  and  product-component 
requirements. 
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Requirements  Development  2 

The  purpose  of  requirements  development  is  to  produce 
and  analyze  customer,  product,  and  product-component 
requirements. 

For  Acquisition,  requirements  development  has  two 
contexts; 

•  The  amalgamation  and  coordination  of  the  operational 
requirements  (customer  requirements)  into  a  requirements 
set  that  will  define  the  scope  and  direction  of  the  acquisition; 

•  The  allocation  and  extension  of  the  customer  requirements 
and  additional  acquirer  requirements  (e.g.,  architecture, 
formal  and  informal  reviews,  reporting  or  data  requirements) 
that  become  the  basis  of  the  processes  utilized  by  the 
supplier’s  organization. 
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There  is  a  continuous  iteration  of  requirements  down  through 
the  multiple  tiers  of  requirements  documents  associated  with  the 
components  of  the  system. 

•  For  example,  requirements  flow  from  the  stakeholders  to  the 
system  level  to  multiple  subsystem  levels  and  eventually  to 
either  hardware  or  software  component  levels. 

The  responsibility  for  developing  requirements  across  the  levels 
is  generally  split  between  the  acquirer  and  the  supplier. 

•  The  acquirer  is  generally  responsible  for  the  higher  level, 
starting  with  operational  requirements  and  the  supplier  is 
responsible  for  successive  levels  below  that. 
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Symptoms 

•  Unstated  requirements  or  poorly  stated  requirements  lead  to 
confusion  among  staff  and  customers. 

•  Design,  implementation,  and  test  work  products  inconsistently 
interpret  the  requirements. 

•  It  takes  a  long  time  to  get  agreement  on  product  design. 

Why  should  we  care? 

•  Unusable  products  and  unhappy  customers 

•  Wasted  time  and  resources  building  the  “wrong”  product 

•  Staff  members  get  tired  of  rework  because  requirements  have 
been  re-interpreted  yet  again. 

•  Excessive  spending  to  satisfy  customer  expectations 


©  2005  by  Carnegie  Mellon  University 


CMMI-AM  and  Engineering  vO.1 


CMMI  Acquisition  Module  -  Page  M3-10 


Carnegie  Mel  Ion 

Software  Engineering  Institute 


Requirements:  Input  or  Output? 


You  RECEIVE  requirements  from  your  customer 
•  Operational  needs 


You  DELIVER  requirements  to  your  supplier 

•  Through  the  solicitation,  SOW,  SOO,  and/or  contract 

Your  job  is 

•  To  ensure  the  quality  of  the  inputs 

•  To  convert  the  inputs  to  the  high-quality  outputs 
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Requirements  and  DoD  Acquisition  ^ 
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Requirements  and  DoD  Acquisition  2 


User  input 


User 

validation 


Requirements 

Development 

RFP,  SOW, 
SOO,  etc. 


Analyze  requirements 
Derive  requirements 
Allocate  requirements 


Component 


PMO )  Supplier 


Sub- 
systems 


Component 


Component 
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Develop  Customer  Requirements 


•  Collect  stakeholder  needs 

•  Elicit  needs 

•  Develop  customer  requirements 


Typical  Work  Products 

•Customer  requirements 

•Customer  constraints 
on  the  conduct  of 
verification 

•Customer  constraints 
on  the  conduct  of 
validation 


Sample  Key  Issues 


Overly  simplistic  elicitation 


Failure  to  identify  /  validate 
“root”  of  the  requirement 


Lack  of  attention  to 
stakeholder  requirements 


Failure  to  involve  all 
stakeholders 
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Develop  Product  Requirements 


CMMI 


•  Establish  product  and  product  component  requirements 

•  Allocate  requirements 

•  Identify  interface  requirements 


Typical  Work  Products 

•  Product  and  Product 
component  requirements 

•  Derived  requirements 

•  Requirement  allocation 
sheets 

•  Design  constraints 

•  Interface  requirements 

•  Relationships  among 
derived  requirements 


Sample  Key  Issues 


Failure  to  address  critical  product 
qualities  &  performance 

Insufficient  specification  &  under¬ 
standing  of  interface  requirements 

Forgetting  to  allocate  all  require¬ 
ments,  including  derived  requirements 

Failure  to  derive  requirements  based 
upon  selection  of  a  technology 
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Requirements  Must  be  Balanced 
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Analyze  and  Validate  Requirements 


CMMI 


•  Establish  operational  concepts  and  scenarios 

•  Establish  a  definition  of  required  functionality 

•  Analyze  requirements  to  achieve  balance 

•  Validate  requirements 


Typical  Work  Products 

•  Operational  concept 

•  Product  installation,  operational, 
maintenance  and  support  concepts 

•  Use  cases  and  activity  diagrams 

•  Requirements  defects  reports 

•Technical  performance  measures 

•  New  requirements  and  proposed 
changes  to  resolve  defects 

•  Assessment  of  risks  related  to 
requirements 


Sample  Key  Issues 


Insufficiently  detailed 
Operational  Concept 


Not  verifying  requirements  are 
complete,  feasible,  realizable 
and  verifiable 


Missing  required  balance  in 
requirements 


Failure  to  use  multiple 
techniques  to  validate  the 
product  will  perform  in  the 
user’s  environment 
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Requirements  Must  Evolve  i 


TIME  ►  [EPIC  2002] 
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Requirements  Must  Evolve  2 
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Requirements  Must  Evolve  3 
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Requirements  Development 

CMMI-AM  Goals  and  Practices  i 


Specific  Goal  Specific  Practice 


Develop 

Customer 

Requirements 

Elicit  Needs 

Develop  the  Customer  Requirements 

Develop 

Product 

Requirements 

Establish  Product  and  Product-  Component 
Requirements 

Allocate  Product-Component  Requirements 

Identify  Interface  Requirements 

Analyze  and 

Validate 

Requirements 

Establish  Operational  Concepts  and  Scenarios 
Establish  a  Definition  of  Required  Functionality 
Analyze  Requirements 

Analyze  Requirements  to  Achieve  Balance 

Validate  Requirements  with  Comprehensive 

Methods 
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Requirements  Development 

Goal  1 :  Develop  Customer  Req’ts  i 

Stakeholder  needs,  expectations,  constraints,  and  interfaces 
are  collected  and  translated  into  customer  requirements 


Elicit  stakeholder  needs,  expectations,  constraints,  and 
interfaces  for  all  phases  of  the  product  life  cycle 

•  Identify  stakeholders  who  are  authorized  to  provide 
requirements 

•  Be  sure  to  include  users,  testers,  and  maintainers 

•  Document  all  gathered  information,  including  the  source  of  the 
requirement. 

•  Maintain  involvement  of  the  stakeholders  throughout  the 
acquisition 
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Requirements  Development 

Goal  1 :  Develop  Customer  Req’ts  2 


CMMI 


Transform  stakeholder  needs  expectations,  constraints,  and 
interfaces  into  customer  requirements 

•  Transform  into  engineering-oriented  requirements 

•  Ensure  requirements  are  complete,  consistent,  clear,  and 
unambiguous. 

•  Document  traceability  to  the  elicited  need 

•  Don’t  forget  non-functional  requirements  such  as  reliability, 
maintainability,  expandability,  etc. 

•  Capture  requirements  for  external  interfaces 

-Particularly  important  for  systems-of-systems 
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Requirements  Development 

Goal  2:  Develop  Product  Req’ts  i 

Customer  requirements  are  refined  and  elaborated  to 
develop  product  and  product-component  requirements _ 

Establish  and  maintain  product  and  product-component 
requirements  which  are  based  on  the  customer  requirements 

•  Part  of  this  is  your  job.  Part  of  it  is  your  supplier’s 

•  New  requirements  will  be  DERIVED  from  existing  ones 

•  Ensure  requirements  are  complete,  consistent,  clear,  and 
unambiguous. 

•  Don’t  forget  non-functional  requirements  such  as  reliability, 
maintainability,  expandability,  etc. 

•  Maintain  traceability 

Allocate  the  requirements  for  each  product  component 

•  Make  sure  that  ALL  requirements  are  allocated  somewhere 
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Requirements  Development 

Goal  2:  Develop  Product  Req’ts  2 

Identify  interface  requirements 

•  Develop  requirements  for  external  interfaces 

-  Particularly  important  for  systems-of-systems 

•  Develop  requirements  for  internal  interfaces 

-  Particularly  important  to  ensure  maintainability,  modifiability, 
expandability,  etc. 
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Requirements  Development 

Goal  3:  Analyze  and  Validate  i 

The  requirements  are  analyzed  and  validated,  and  a 
definition  of  required  functionality  is  developed 

Establish  and  maintain  operational  concepts  and  associated 
scenarios 

•  Captures  the  inputs  of  your  customer 

•  Used  as  a  basis  for  requirements  development  and 
requirements  validation 

Establish  and  maintain  a  definition  of  required  functionality 

•  Necessary  to  manage  expectations  of  stakeholders 

•  Keep  it  up  to  date  and  widely  available 

Analyze  requirements  to  ensure  that  they  are  necessary  and 
sufficient 

•  Check  for  quality  of  the  requirements 

-Complete,  consistent,  clear,  unambiguous 

•  Check  for  traceability  (forward  and  backward) 
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Requirements  Development 

Goal  3:  Analyze  and  Validate  2 


CMMI 


Analyze  requirements  to  balance  stakeholder  needs  and 
constraints 

•  Trades  between 

•  Obtain  stakeholder  buy-in  on  trades 

-  you’ll  regret  it  later  if  you  don’t 


Validate  requirements  to  ensure  the  resulting  product  will 
perform  as  intended  in  the  user’s  environment  using  multiple 
techniques  as  appropriate 

•  Key  element  to  obtain  stakeholder  buy-in 
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Agenda 


CMMI 


Engineering  Process  Areas 
•  Requirements  Development 


•  Requirements  Management 


•  Verification 

•  Validation 
Summary 
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Requirements  Management  i 

The  purpose  of  requirements  management  is  to 
manage  the  requirements  of  the  project’s  products  and 
product  components  and  to  identify  inconsistencies 
between  those  requirements  and  the  project’s  plans  and 
work  products. 

For  Acquisition,  requirements  management  is  applied  to 
the  requirements  that  are  received  from  the 
requirements  development  process. 
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Requirements  Management  2 

During  acquisition,  requirements  management  includes 

•  the  direct  management  of  acquirer-controlled 
requirements 

•  oversight  of  supplier  requirements  management 

Requirements  are  managed  and  maintained  with 
discipline  so  that  changes  are  not  executed  without 
recognizing  the  impact  to  the  project. 

Requirements  management  does  not  end  with  the 
selection  of  a  supplier  and  an  award. 

•  The  acquisition  project  continues  to  manage  high-level 
requirements,  including  changes 

•  the  selected  supplier  manages  the  lower  level 
requirements 
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Poor  Requirements  Management ... 

Symptoms 

•  High  levels  of  re-work  throughout  the  project. 

•  Requirements  are  accepted  by  staff  from  any  unauthorized  sources. 

•  “Galloping”  requirements  creep. 

•  Inability  to  prove  that  the  product  meets  the  approved  requirements 

Why  should  we  care? 

•  Solutions  that  don’t  match  user  needs  or  may  have  to  be  replaced  or 
retired  early 

•  Inability  to  hold  contractor  to  commitments 

•  Excessive  budget  consumption  [LEFF  2003] 

-  Requirements  errors  are  the  most  common  error  &  most  expensive 
to  fix 

-  Requirements  error  are  likely  to  consume  25%  -  40%  of  the  total 
project  budget  when  not  caught  early 
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Obtain  an  Understanding  of  Reqts. 


•  Establish  criteria  for  distinguishing  appropriate  requirements 

•  Establish  objective  criteria  for  the  acceptance  of  requirements 

•  Analyze  requirements  to  ensure  that  the  criteria  are  met 

•  Reach  an  understanding  of  the  requirements 
with  the  stakeholders 


Typical  Work  Products 

•  Lists  of  criteria  for 
distinguishing  appropriate 
requirement  providers 

•  Criteria  for  evaluation  and 
acceptance  of  requirements 

•  Results  of  analyses  against 
criteria 

•Agreed  to  set  of  requirements 


Sample  Key  Issues 


Missing  stakeholders 
Lack  of  acceptance  criteria 
resulting  in  inadequate 
verification,  rework  or  system 
rejection 

Failure  to  have  a  common 
understanding  of  requirements 
^Insufficient  analysis  techniques^ 
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Obtain  Commitment  to  Requirements 


•  Assess  the  impact  of  requirements  on  existing 
commitments 

•  Negotiate  and  record  commitments 


Typical  Work  Products  Sample  Key  Issues 

•  Requirement  impact 
assessments 

•  Documented 
commitments  to 
requirements  and 
requirement  changes 


Inadequate  assessments 
Existing  commitments  are  not 
well  known  or  defined 
Failure  to  negotiate  with 
balance  in  mind 
Failure  to  obtain  written 
commitment 
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Manage  Requirements  Changes 


CMMI 


•  Capture  all  requirements  and  requirements  changes 

•  Maintain  the  requirements  change  history  with  the  rationale  for 
the  changes 

•  Evaluate  the  impact  of  the  requirement  changes  from  the 
standpoint  of  the  stakeholders 

•  Make  the  requirements  and  change  data  available 


Typical  Work  Products 

•Requirement  impact 
assessments 

•Documented  commitments 
to  requirements  and 
requirement  changes 


Sample  Key  Issues 

Lagging  documentation 
Failure  to  plan  for  and  manage 
the  change  process 
Incomplete  impact  assessments 
Lack  of  backup  plans 
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Requirements  Management 

CMMI-AM  Goals  and  Practices  i 


Specific  Goal  Specific  Practice 


Manage 

Requirements 


Obtain  an  Understanding  of  Requirements 

Obtain  Commitment  to  Requirements 

Manage  Requirement  Changes 

Maintain  Bidirectional  Traceability  of 
Requirements 

Identify  Inconsistencies  Between 
Project  Work  and  Requirements 
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Requirements  Management 

Goal  1:  Manage  Requirements  i 

Requirements  are  managed  and  inconsistencies  with  project 
plans  and  work  products  identified 

Develop  an  understanding  with  the  requirements  providers 
on  the  meaning  of  the  requirements 

•  Be  proactive  to  fully  understand  the  intentions  of  the  users 

•  Identify  the  “root  source”  of  the  requirement 

Obtain  commitment  to  the  requirements  from  the  project 
participants 

•  Written  commitments  from  users,  maintainers,  testers,  etc. 
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Requirements  Management 

Goal  1:  Manage  Requirements  2 


CMMI 


Manage  changes  to  the  requirements  as  they  evolve  during 
the  project 

•  Apply  rigorous  configuration  management 

•  Establish  a  change  control  board 

•  Establish  a  process  to  evaluate  impact  of  proposed  changes 
on  cost,  schedule,  and  performance 


Maintain  bi-directional  traceability  among  the  requirements 
and  the  project  plans  and  work  products 

•  Ensures  that  all  higher-level  requirements  are  addressed 

•  Ensures  that  all  lower-level  requirements  arise  from  a  parent 
requirement 

•  Indispensable  for  test  planning 

Identify  inconsistencies  between  the  project  plans  and  work 
products  and  the  requirements 
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Agenda 


CMMI 


•  ® 


Engineering  Process  Areas 

•  Requirements  Development 

•  Requirements  Management 

•  Verification 

•  Validation 
Summary 
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Verification  versus  Validation 


CMMI 


Verification 

•  Are  you  building  the  product  right ? 

•  That  is,  are  you  meeting  the  specified  requirements? 


Validation 

•  Are  you  building  the  right  product ? 

•  That  is,  are  you  meeting  the  operational  need? 
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Verification 

The  purpose  of  verification  is  to  ensure  that  selected  work 
products  meet  their  specified  requirements. 

For  Acquisition,  verification  involves  ensuring  that  the  evolving 
work  products  of  the  acquisition  project  meet  specified 
requirements  for  those  products.  The  acquisition  project  should 
ensure 

•  a  proper  verification  environment  exists 

•  work  products  are  selected  for  evaluation  based  on 
documented  criteria. 

Peer  reviews  are  intended  to  be  used  for  work  products 
developed  by  the  acquisition  project 

The  acquisition  project  is  also  responsible  for  ensuring  that  the 
supplier  uses  appropriate  methods  to  verify  its  work  products. 


©  2005  by  Carnegie  Mellon  University 


CMMI-AM  and  Engineering  vO.1 


CMMI  Acquisition  Module  -  Page  M3-40 


Carnegie  Mel  Ion 

Software  Engineering  Institute 


Poor  Verification  ... 


CMMI 


Symptoms 

•  There  is  disagreement  among  technical  staff  as  to  the  “done¬ 
ness”  of  different  components. 

•  Under  test  the  product  doesn’t  meet  requirements  or  design 
expectations. 

•  Defects  that  could  have  been  caught  early  escape  into  later  life 
cycle  phases. 

•  There  is  increased  integration  or  test  time. 

Why  should  we  care? 

•  Product  reliability  suffers  if  defects  aren’t  detected  or  corrected 
prior  to  customer  release. 

•  The  product  costs  more  to  test  if  early  verification  activities  are 
ignored. 

•  Customers  don’t  want  to  pay  for  defective  products,  and  you 
probably  won’t  get  their  business  next  time 
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Verification  i 

Activities  to  evaluate  progress  and  effectiveness  of 
evolving  system  products  and  processes  and  to 
measure  specification  compliance 

Questions  the  PMO  needs  to  address: 

•  Are  the  software  test  plans  adequate? 

•  Has  combined  DT/OT  testing  been  considered? 

•  Is  the  software/hardware  integration  testing  rigorous? 

Questions  the  contractor  needs  to  address: 

•  Is  there  traceability  from  requirements  to  testing? 

•  Is  testing  started  early  and  at  the  lowest  possible  level? 

•  Has  independent  quality  assurance  been  involved  in  the 
test  process  from  early  in  the  program? 
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Verification  2 

Software  testing  poses  new  challenges 

•  Various  types  of  software  products  to  be  tested 

•  Many  levels/types  of  software  testing  to  be  considered 

•  Software  testing  often  focused  on  functional 
requirements  only  -  don’t  neglect  non-functional 
requirements 

•  Software  testing  may  require  specific  hardware,  software 
and  personnel  test  assets  -  PLAN  AHEAD! 
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Types  of  Software  to  be  Tested 


System  software 
User  interfaces 

Modeling  and  simulation  software 
Training  software 
Communications  interfaces 
COTS  products/interfaces 
Programming  Interfaces  (extensions) 


©  2005  by  Carnegie  Mellon  University  CMMI-AM  and  Engineering  vO.1 


CMMI  Acquisition  Module  -  Page  M3-44 


Carnegie  Mel  Ion 

Software  Engineering  Institute 

Levels  of  Software  Testing 


Peer  reviews 
Unit  test 

Integration  testing 
Interoperability  testing 
Security  testing 
Usability  testing 


System  testing 
Acceptance  test 
Fail-back  testing 
Operational  testing 
Regression  testing 
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Robustness  Testing 


Software  testing  often  focuses  on  functional  requirements 
only 

“Happy  Path”  testing  is  often  the  norm;  “robustness” 
testing  often  overlooked 

Many  areas  require  robustness  testing 

•  Load  •  Interfaces  to  other  systems 

•  Stress  •  Conversions/limits/corner 


conditions 


Security 


•  Error  detection  and  correction  *  Timing/Synchronization 


User  inputs 

Graphical  User  Interface 
(GUI)  structure 


Memory  usage 
Path  coverage 
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Test  Support  Needs 


Software  testing  may  require  specific  hardware, 
software  test  assets 

•  Prototype  hardware  for  early  software  testing 

•  Special  test  fixtures  to  test  error  conditions 

•  Interface  mock-ups  or  emulators 

•  Special  software  for  monitoring  system  performance 

•  Special  software  to  test  “abnormal”  system  conditions 

Software  testing  may  require  specific  test  personnel 

•  Test  team  independent  of  the  design  team 

•  Users  available  for  user  interface  testing 
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Verification 

CMMI-AM  Goals  and  Practices  i 


Specific  Goal 

Specific  Practice 

Prepare  for 
Verification 

•  Select  Work  Products  for  Verification 

•  Establish  the  Verification  Environment 

•  Establish  Verification  Procedures 
and  Criteria 

Perform  Peer 
Reviews 

•  Prepare  for  Peer  Reviews 

•  Conduct  Peer  Reviews 

•  Analyze  Peer  Review  Data 

Verify  Selected  •  Perform  Verification 

Work  Products  #  Analyze  Verification  Results  and  Identify  Corrective 

Action 
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Verification 

Goal  1 :  Prepare  for  Verification 

Preparation  for  verification  is  conducted 

Select  the  work  products  to  be  verified  and  the  verification 
methods  that  will  be  used  for  each 

•  Include  work  products  of  the  acquirer  as  well  as  the  supplier 

•  Typical  verification  methods  include  peer  review, 
demonstration,  inspection,  testing,  and  analysis 

Establish  and  maintain  the  environment  needed  to  support 
verification 

•  Plan  for  staff,  training,  equipment,  and  facility  needs 

Establish  and  maintain  verification  procedures  and  criteria 
for  the  selected  work  products 

•  Maintain  test  procedures  and  acceptance  criteria  under 
configuration  management 

•  Ensure  traceability  to  requirements 
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Verification 

Goal  2:  Perform  Peer  Reviews 


CMMI 


Peer  reviews  are  performed  on  selected  work  products 

Prepare  for  peer  reviews  of  selected  work  products 

•  Establish  a  peer  review  procedure. 

•  T rain  the  staff 


Conduct  peer  reviews  on  selected  work  products  and 
identify  issues  resulting  from  the  peer  review 

•  Notify  reviewers  and  distribute  materials  for  review  far  enough 
in  advance  to  enable  comprehensive  review 

Analyze  data  about  preparation,  conduct,  and  the  results  of 
the  peer  reviews 

•  Document  results  of  peer  review 

•  Assign  responsibility  and  due  dates  for  all  actions 
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CMMI 


Goal  3:  Verify  Work  Products  i 

Selected  work  products  are  verified  against  their 
specified  requirements 


® 


Perform  verification  on  the  selected  work  products 

•  Verify  using  approved  procedures  and  acceptance  criteria 

•  Document  all  test  results  (pass  or  fail) 


Analyze  the  results  of  all  verification  activities  and  identify 
corrective  action 

•  Document  analysis  and  corrective  actions 

•  Assign  responsibility  and  due  dates  for  all  corrective  actions 
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Agenda 


CMMI 


•  ® 


Engineering  Process  Areas 

•  Requirements  Development 

•  Requirements  Management 

•  Verification 

•  Validation 
Summary 
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Validation  i 


CMMI 


The  purpose  of  Validation  is  to  demonstrate  that  a  product  or 
product  component  fulfills  its  intended  use  when  placed  in  its 
intended  environment. 

Validation  activities  can  be  applied  to  all  aspects  of  the  product  in 
any  of  its  intended  environments 

•  e.g.,  operation,  training,  manufacturing,  maintenance,  and 
support  services. 

The  methods  employed  to  accomplish  validation  can  be  applied 

to  work  products  as  well  as  to  the  product  and  product 
components. 

The  work  products  (e.g.,  requirements,  designs,  prototypes) 
should  be  selected  for  validation  based  on  which  are  the  best 
predictors  of  how  well  the  delivered  end  product  and  product 
components  will  satisfy  user  needs. 
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Validation  2 

For  acquisition,  validation  involves  ensuring  that  the  evolving 
acquisition  work  products  (e.g.,  RFPs,  SOWs,  plans)  meet  the 
acquisition  project’s  needs. 

Validation  activities  are  normally  performed  early  and 
continuously  throughout  the  acquisition  life  cycle. 

The  acquirer  also  uses  validation  processes  to  ensure  that  the 
product  or  service  received  from  the  supplier  will  fulfill  its  intended 
use. 

The  test  community  is  a  major  stakeholder,  participating  in  up¬ 
front  planning  through  final-product  acceptance. 

•  The  supplier  and/or  the  test  community  may  perform  many  of 
the  validation  practices,  with  the  acquisition  project  facilitating 
the  correction  of  deficiencies  or  enhancements  by  the  supplier 
or  follow-on  maintenance  organization. 
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Symptoms 

•  Lots  of  user  change  requests  are  received  before  or  soon 
after  the  product  is  released. 

•  There  are  arguments  among  the  technical  staff  as  to 
what  the  user  really  wants. 

•  The  released  product  doesn’t  meet  user  expectations. 

Why  should  we  care? 

•  Customers  don’t  want  to  pay  for  products  that  don’t  meet 
their  needs. 

•  If  an  end  user  refuses  to  use  the  product  as  delivered, 
their  confidence  in  you  is  eroded. 

•  You’ll  spend  a  lot  of  money  trying  to  make  it  right,  or 
you’ll  give  up  that  customer’s  future  business. 
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Validation 

CMMI-AM  Goals  and  Practices  i 


Specific  Goal 

Specific  Practice 

Prepare  for 
Validation 

•  Select  Products  for  Validation 

•  Establish  the  Validation  Environment 

•  Establish  Validation  Procedures 
and  Criteria 

Validate 
Product  of 
Product 
Components 

•  Perform  Validation 

•  Analyze  Validation  Results 
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Validation 

Goal  1:  Perform  Validation  i 


•  ® 


Preparation  for  validation  is  conducted 


Select  products  and  product  components  to  be  validated  and 
the  validation  methods  that  will  be  used  for  each 

•  Acquisition  Strategy  •  SOW 

•  Supplier-developed  requirements  •  Test  plans 

•  Delivered  end  product  •  Training  materials 

•  Support  and  maintenance  plans  •  etc. 


Establish  and  maintain  the  environment  needed  to  support 
validation 

•  Plan  for  staff,  training,  equipment,  and  facility  needs 

Establish  and  maintain  procedures  and  criteria  for  validation 

•  Maintain  validation  procedures  and  acceptance  criteria  under 
configuration  management 

•  Ensure  traceability  to  operational  requirements 
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Goal  2:  Prepare  for  Validation 

The  product  or  product  components  are  validated  to 
ensure  that  they  are  suitable  for  use  in  their  intended 
operating  environment 

Perform  validation  on  selected  products  and  product 
components 

•  Verify  using  approved  procedures  and  acceptance  criteria 

•  Document  all  test  results  (pass  or  fail) 

Analyze  the  results  of  the  validation  activities  and  identify 
issues 

•  Document  analysis  and  corrective  actions 

•  Assign  responsibility  and  due  dates  for  all  corrective  actions 
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Agenda 

Engineering  Process  Areas 

•  Requirements  Development 

•  Requirements  Management 

•  Verification 

•  Validation 
Summary 


CMMI 
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PMO  plays  a  critical  role  in  the  systems  engineering  of  a  project 

Principal  goals  of  Requirements  Development 

•  Develop  Customer  Requirements 

•  Develop  Product  Requirements 

•  Analyze  and  Validate  Requirements 

Principal  goals  of  Requirements  Management 

•  Manage  Requirements 
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Summary  2 

Principal  goals  of  Verification 

•  Prepare  for  Verification 

•  Perform  Peer  Reviews 

•  Verify  Selected  Work  Products 

Principal  goals  of  Validation 

•  Prepare  for  Validation 

•  Validate  Product  of  Product  Components 
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